Center for Research Computing, University of Notre Dame
2023-09-05
Building Stuff?
Building Agents based on Large Language Models!
Jeremy Jordan. “Effective Testing for Machine Learning Systems.,” August 19, 2020. https://www.jeremyjordan.me/testing-ml/.
“An autoregressive large language model (AR-LLM) is a type of neural network model that can generate natural language text. It has a very large number of parameters (billions or trillions) that are trained on a huge amount of text data from various sources. The main goal of an AR-LLM is to predict the next word or token based on the previous words or tokens in the input text. For example, if the input text is”The sky is”, the AR-LLM might predict “blue” as the next word. AR-LLMs can also generate text from scratch by sampling words from a probability distribution. For example, if the input text is empty, the AR-LLM might generate “Once upon a time, there was a princess who lived in a castle.” as the output text.”1
Schuurmans, Dale. “Memory Augmented Large Language Models Are Computationally Universal.”
arXiv, January 9, 2023. https://doi.org/10.48550/arXiv.2301.04589.
We will focus on Conversational Agents…
Andrej Karpathy, “State of GPT” | BRK216HFS, Microsoft Build, 2023.
Andrej Karpathy, “State of GPT” | BRK216HFS, Microsoft Build, 2023.
“Prompt engineering is the process of designing and refining the prompts or input stimuli for a language model to generate specific types of output. Prompt engineering involves selecting appropriate keywords, providing context, and shaping the input in a way that encourages the model to produce the desired response and is a vital technique to actively shape the behavior and output of foundation models.”1
Ouyang, Long, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, et al.
“Training Language Models to Follow Instructions with Human Feedback.”
arXiv, March 4, 2022. https://doi.org/10.48550/arXiv.2203.02155.
Kojima, Takeshi, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa.
“Large Language Models Are Zero-Shot Reasoners.” arXiv, January 29, 2023.
Henry Zeng, Lauryn Gayhardt, Jill Grant “What is Azure Machine Learning prompt flow
(preview) - Azure Machine Learning,” Jul. 02, 2023.
http://tiny.cc/kelavz (accessed Sep. 04, 2023).
Henry Zeng, Lauryn Gayhardt, Jill Grant “What is Azure Machine Learning prompt flow
(preview) - Azure Machine Learning,” Jul. 02, 2023.
http://tiny.cc/kelavz (accessed Sep. 04, 2023).
Besta, Maciej, Nils Blach, Ales Kubicek, Robert Gerstenberger, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, et al
“Graph of Thoughts: Solving Elaborate Problems with Large Language Models.” arXiv, August 21, 2023.
“Prompt Engineering Guide – Nextra.”
https://www.promptingguide.ai/ (accessed Sep. 04, 2023).
“Foundation models are usually trained offline, making the model agnostic to any data that is created after the model was trained. Additionally, foundation models are trained on very general domain corpora, making them less effective for domain-specific tasks. You can use Retrieval Augmented Generation (RAG) to retrieve data from outside a foundation model and augment your prompts by adding the relevant retrieved data in context. For more information about RAG model architectures”^[“Retrieval Augmented Generation (RAG) - Amazon SageMaker.” Accessed September 4, 2023. http://tiny.cc/f3mavz.
“Custom Retriever Combining KG Index and VectorStore Index
S. Patil, “Gorilla: Large Language Model Connected with Massive APIs [Project Website].” Sep. 04, 2023.
Accessed: Sep. 04, 2023. [Online]. Available: https://github.com/ShishirPatil/gorilla
S. Patil, “Gorilla: Large Language Model Connected with Massive APIs [Project Website].” Sep. 04, 2023.
Accessed: Sep. 04, 2023. [Online]. Available: https://github.com/ShishirPatil/gorilla
JSON-Grammar
root ::= object
value ::= object | array | string | number | ("true" | "false" | "null") ws
object ::=
"{" ws (
string ":" ws value
("," ws string ":" ws value)*
)? "}" ws
array ::=
"[" ws (
value
("," ws value)*
)? "]" ws
string ::=
"\"" (
[^"\\] |
"\\" (["\\/bfnrt] | "u" [0-9a-fA-F] [0-9a-fA-F] [0-9a-fA-F] [0-9a-fA-F]) # escapes
)* "\"" ws
number ::= ("-"? ([0-9] | [1-9] [0-9]*)) ("." [0-9]+)? ([eE] [-+]? [0-9]+)? ws
# Optional space: by convention, applied in this grammar after literal chars when allowed
ws ::= ([ \t\n] ws)?
“speculative : add grammar support by ggerganov · Pull Request #2991 · ggerganov/llama.cpp,”
GitHub. https://github.com/ggerganov/llama.cpp/pull/2991 (accessed Sep. 04, 2023).
Andrej Karpathy, “State of GPT” | BRK216HFS, Microsoft Build, 2023.
Matt Bronstein and Rajko Radovanovic, “Supporting the Open Source AI Community,”
Andreessen Horowitz, Aug. 30, 2023.
http://tiny.cc/uflavz (accessed Sep. 03, 2023).